Removing the Barrier to Scalability in Parallel FMM

The Fast Multipole Method (FMM) is well known to possess a bottleneck arising from decreasing workload on higher levels of the FMM tree [Greengard and Gropp, Comp. Math. Appl., 20(7), 1990]. We show that this potential bottleneck can be eliminated by overlapping multipole and local expansion computations with direct kernel evaluations on the finest level grid.Comment: 11 pages, 2 figure

Modeling Charge-Sign Asymmetric Solvation Free Energies With Nonlinear Boundary Conditions

We show that charge-sign-dependent asymmetric hydration can be modeled accurately using linear Poisson theory but replacing the standard electric-displacement boundary condition with a simple nonlinear boundary condition. Using a single multiplicative scaling factor to determine atomic radii from molecular dynamics Lennard-Jones parameters, the new model accurately reproduces MD free-energy calculations of hydration asymmetries for (i) monatomic ions, (ii) titratable amino acids in both their protonated and unprotonated states, and (iii) the Mobley "bracelet" and "rod" test problems [J. Phys. Chem. B, v. 112:2408, 2008]. Remarkably, the model also justifies the use of linear response expressions for charging free energies. Our boundary-element method implementation demonstrates the ease with which other continuum-electrostatic solvers can be extended to include asymmetry.Comment: 7 pages, 2 figures, accepted to Journal of Chemical Physic

Support for Non-conformal Meshes in PETSc's DMPlex Interface

PETSc's DMPlex interface for unstructured meshes has been extended to support non-conformal meshes. The topological construct that DMPlex implements---the CW-complex---is by definition conformal, so representing non- conformal meshes in a way that hides complexity requires careful attention to the interface between DMPlex and numerical methods such as the finite element method. Our approach---which combines a tree structure for subset- superset relationships and a "reference tree" describing the types of non-conformal interfaces---allows finite element code written for conformal meshes to extend automatically: in particular, all "hanging-node" constraint calculations are handled behind the scenes. We give example code demonstrating the use of this extension, and use it to convert forests of quadtrees and forests of octrees from the p4est library to DMPlex meshes.Comment: 16 pages, 13 figures, 5 code example

Work/Precision Tradeoffs in Continuum Models of Biomolecular Electrostatics

The structure and function of biological molecules are strongly influenced by the water and dissolved ions that surround them. This aqueous solution (solvent) exerts significant electrostatic forces in response to the biomolecule's ubiquitous atomic charges and polar chemical groups. In this work, we investigate a simple approach to numerical calculation of this model using boundary-integral equation (BIE) methods and boundary-element methods (BEM). Traditional BEM discretizes the protein--solvent boundary into a set of boundary elements, or panels, and the approximate solution is defined as a weighted combination of basis functions with compact support. The resulting BEM matrix then requires integrating singular or near singular functions, which can be slow and challenging to compute. Here we investigate the accuracy and convergence of a simpler representation, namely modeling the unknown surface charge distribution as a set of discrete point charges on the surface. We find that at low resolution, point-based BEM is more accurate than panel-based methods, due to the fact that the protein surface is sampled directly, and can be of significant value for numerous important calculations that require only moderate accuracy, such as the preliminary stages of rational drug design and protein engineering.Comment: 10 pages, 8 figures, in Proceedings of ASME 2015 International Mechanical Engineering Congress & Exposition, 201

Multiscale models and approximation algorithms for protein electrostatics

Electrostatic forces play many important roles in molecular biology, but are hard to model due to the complicated interactions between biomolecules and the surrounding solvent, a fluid composed of water and dissolved ions. Continuum model have been surprisingly successful for simple biological questions, but fail for important problems such as understanding the effects of protein mutations. In this paper we highlight the advantages of boundary-integral methods for these problems, and our use of boundary integrals to design and test more accurate theories. Examples include a multiscale model based on nonlocal continuum theory, and a nonlinear boundary condition that captures atomic-scale effects at biomolecular surfaces.Comment: 12 pages, 6 figure

Unstructured Overlapping Mesh Distribution in Parallel

We present a simple mathematical framework and API for parallel mesh and data distribution, load balancing, and overlap generation. It relies on viewing the mesh as a Hasse diagram, abstracting away information such as cell shape, dimension, and coordinates. The high level of abstraction makes our interface both concise and powerful, as the same algorithm applies to any representable mesh, such as hybrid meshes, meshes embedded in higher dimension, and overlapped meshes in parallel. We present evidence, both theoretical and experimental, that the algorithms are scalable and efficient. A working implementation can be found in the latest release of the PETSc libraries.Comment: 14 pages, 6 figures, submitted to TOM

Run-time extensibility and librarization of simulation software

Build-time configuration and environment assumptions are hampering progress and usability in scientific software. That which would be utterly unacceptable in non-scientific software somehow passes for the norm in scientific packages. The community needs reusable software packages that are easy use and flexible enough to accommodate next-generation simulation and analysis demands.Comment: 6 page

Flexible, Scalable Mesh and Data Management using PETSc DMPlex

Designing a scientific software stack to meet the needs of the next-generation of mesh-based simulation demands, not only scalable and efficient mesh and data management on a wide range of platforms, but also an abstraction layer that makes it useful for a wide range of application codes. Common utility tasks, such as file I/O, mesh distribution, and work partitioning, should be delegated to external libraries in order to promote code re-use, extensibility and software interoperability. In this paper we demonstrate the use of PETSc's DMPlex data management API to perform mesh input and domain partitioning in Fluidity, a large scale CFD application. We demonstrate that raising the level of abstraction adds new functionality to the application code, such as support for additional mesh file formats and mesh re- ordering, while improving simulation startup cost through more efficient mesh distribution. Moreover, the separation of concerns accomplished through this interface shifts critical performance and interoperability issues, such as scalable I/O and file format support, to a widely used and supported open source community library, improving the sustainability, performance, and functionality of Fluidity.Comment: 6 pages, 6 figures, to appear in EASC 201

Generalizing The Mean Spherical Approximation as a Multiscale, Nonlinear Boundary Condition at the Solute--Solvent Interface

In this paper we extend the familiar continuum electrostatic model with a perturbation to the usual macroscopic boundary condition. The perturbation is based on the mean spherical approximation (MSA), to derive a multiscale hydration-shell boundary condition (HSBC). We show that the HSBC/MSA model reproduces MSA predictions for Born ions in a variety of polar solvents, including both protic and aprotic solvents. Importantly, the HSBC/MSA model predicts not only solvation free energies accurately but also solvation entropies, which standard continuum electrostatic models fail to predict. The HSBC/MSA model depends only on the normal electric field at the dielectric boundary, similar to our recent development of an HSBC model for charge-sign hydration asymmetry, and the reformulation of the MSA as a boundary condition enables its straightforward application to complex molecules such as proteins.Comment: 14 pages, 2 figure

Finite Element Integration with Quadrature on the GPU

We present a novel, quadrature-based finite element integration method for low-order elements on GPUs, using a pattern we call \textit{thread transposition} to avoid reductions while vectorizing aggressively. On the NVIDIA GTX580, which has a nominal single precision peak flop rate of 1.5 TF/s and a memory bandwidth of 192 GB/s, we achieve close to 300 GF/s for element integration on first-order discretization of the Laplacian operator with variable coefficients in two dimensions, and over 400 GF/s in three dimensions. From our performance model we find that this corresponds to 90\% of our measured achievable bandwidth peak of 310 GF/s. Further experimental results also match the predicted performance when used with double precision (120 GF/s in two dimensions, 150 GF/s in three dimensions). Results obtained for the linear elasticity equations (220 GF/s and 70 GF/s in two dimensions, 180 GF/s and 60 GF/s in three dimensions) also demonstrate the applicability of our method to vector-valued partial differential equations.Comment: 14 pages, 6 figure